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REGIONAL EDUCA TIONAL L ABORATORY 
SOUTHEAST SERVECenter 


EVIDENCE BASED 
EDUCATION 
REQUEST DESK 


OUR GOAL 

To assist educators and 
policymakers in their 
efforts to apply the 
evidence base to 
decisions about policies, 
programs, and practices 
they encounter. 



REQUEST: 

• Early warning indicator systems research for secondary schools 


RESPONSE 

Methodology 

REL-SE staff conducted searches of the Internet and UNCG online education databas- 
es (EconLit, Google Scholar, Wilson Web, etc.) for research and studies regarding the 
state of early warning indicator systems research for secondary schools. REL-SE staff 
also consulted relevant nonprofit organizations (AIR, Mathematica, MDRC, RAND, Ur- 
ban Institute, etc.), state department of education web sites (for AL, FL, GA, MS, NC 
and SC) as well as U.S. government websites (U.S. ED, GAO, lES, WWC, etc.) for in- 
formation. The Additional Resources section provides web sites of interest with specific 
information on dropouts/early warning indicator systems as well as recent conferences 
discussing early warning indicator systems. An annotated bibliography of the research 
follows. 


If you have any questions regarding this document, please contact the 
REL-SE, 1-800-755-3277 or RELSoutheast@serve.org 
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Additional Resources 


1 . American Youth Policy Forum: Improving the Transition from Middle Grades to 
High School; The Role of Early Warning Indicators Lunch Forum, Washington, 
DC, January 25, 2008 

a. Presenters included: Elaine Allensworth (Consortium on Chicago School 
Research, University of Chicago), Robert Balfanz (Johns Hopkins Univer- 
sity), Craig Jerald (Breaking the Curve Consulting) 

b. Three sessions included: 

i. Balfanz, “Keeping Early Secondary Grade Students on the Gradua- 
tion Path: Findings and Policy Implications” 

ii. Allensworth, “What Matters for Staying On-Track and Graduating in 
Chicago Public Schools” 

iii. Jerald, “Stacking the Deck; Using Early Warning Data to Improve 
High School Outcomes” 

c. Session summaries and presenters’ PowerPoint slides are available at: 
http://www.avpf.orq/forumbriefs/2008/fb012508.htm 

2. Center for Public Education: Their section on dropouts provides three nice 
overviews: 

a. Key Lessons About Preventing Dropouts: An at-a-glance guide to re- 
search on dropout identification, intervention, prevention, and recovery. 

b. Research Review: Keeping Kids in School: What research has to tell us 
about preventing dropouts. 

c. Dropouts: Myths v. Realities: Many widespread ideas about dropouts miss 
the mark. 

d. All three overviews can be found at: 
http://www.centerforpubliceducation.Org/site/c.kiJXJ5MPIwE/b.2623519/k. 
DB01 /Dropouts. htm 

3. Everyone Graduates Center: Early warning indicator systems overview: 
http://www.everv1qraduates.orq/ToolsAndModels/EarlvWarninqSvstems.html 

4. National High School Center/BetterHighSchools.org: Dropout and early 
warning materials: 

http://www.betterhiqhschools.com/topics/DropoutWarninqSiqns.asp 


Bibliography 

Achieve and Jobs for the Future (2007 July). Moving North Carolina forward: High 
standards and high graduation rates: A framework for next-generation as- 
sessment and accountability indicators. Washington, DC: Achieve, Inc., and 
Jobs for the Future. 

Abstract: After conducting a thorough policy analysis. Achieve and Jobs for the Future 
suggested in this report to North Carolina ways it can create a more coherent, intention- 
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al, and aligned assessment and accountability system to improve both achievement and 
graduation rates for struggling and out-of-school students. The report includes a prelim- 
inary framework for next-generation high school accountability indicators that recognize 
and reward schools that help all students graduate on time with a college and career- 
ready diploma. The recommendations were presented to an independent Blue Ribbon 
Commission on Testing and Accountability in North Carolina. This report was part of 
Moving Forward: High Standards and High Graduation Rates, a joint project of Achieve, 
Inc., and Jobs for the Future and funded by Carnegie Corporation of New York. [PDF 
included] 

Link: http://www.achieve.orq/files/MovinqNCForward.pdf 


Allensworth, E. M., & Easton, J. Q. (2005 June). The on-track indicator as a predic- 
tor of high schooi graduation. Chicago: Consortium on Chicago Schooi Re- 
search. 

Abstract: This indicator identifies students as on-track if they earn at least five full-year 
course credits and no more than one semester “F” in a core course in their first year of 
high school. On-track students are more than three and one-half times more likely to 
graduate from high school in four years than off-track students. The indicator is a more 
accurate predictor of graduation than students’ previous achievement test scores or 
their background characteristics. Perhaps the most important finding from this report is 
that failures during the first year of high school make a student much less likely to grad- 
uate. Based on their findings, the authors believe that parents and teachers should 
carefully monitor students’ grades, especially in the first semester of freshman year, 
when there are still many opportunities to improve grades. Helping students make a 
successful transition to high school during the first semester could make students more 
likely to graduate. This report also finds that on-track students are not necessarily the 
students with the highest achievement test scores. Many students with strong achieve- 
ment fail to graduate, and many students who have demonstrated weaker achievement 
succeed in graduating. Finally, this report concludes that the particular school a student 
attends plays a large role in whether the student is on-track. While we expect schools to 
have students with differing levels of preparation for high school, differences in the 
number of students on-track at each school remained even when the authors controlled 
for students’ eighth-grade test scores and socioeconomic status. This suggests that 
school climate and structure play a significant role in whether students succeed in high 
school. Schools can use the on-track indicator, which makes use of readily available da- 
ta on course credits and failures, to understand what aspects of the school may be lead- 
ing students to drop out. [PDF/nc/t/c/ec/| 

Link: http://ccsr.uchicaqo.edu/content/publications.php7pub id=1 0 
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Allensworth, E. M., & Easton, J. Q. (2007 July). What matters for staying on-track 
and graduating in Chicago Pubiic Schoois. Chicago: Consortium on Chicago 
School Research. 

Abstract: Almost half of all Chicago Public School students fail to graduate, and in 
some CPS high schools more students drop out than graduate. It is a problem that can 
sometimes feel overwhelming to address because the causes of dropout are myriad 
and complex. What is often lost in discussions about dropping out is the one factor that 
is most directly related to graduation — students’ performance in their courses. In this 
research report, CCSR authors Elaine Allensworth and John Q. Easton look into the 
elements of course performance that predict whether students will graduate and sug- 
gest what schools and families can do to keep more teens in school. Building on earlier 
CCSR research of “on-track indicators” that demonstrated a connection between failing 
freshman classes and dropping out, the authors found that a number of freshman-year 
factors can be used to predict high school graduation. Grades are as predictive as on- 
track indicators; almost all students with a “B” average or better at the end of their 
freshman year graduate compared to only a quarter of those with a “D” average. The 
research also revealed how critical attendance is for freshman success. Conventional 
wisdom holds that eighth-grade test scores are good predictors of students’ likelihood to 
do well in high school courses. However, course attendance is eight times more predic- 
tive of course failure in the freshman year than test scores. Just one week of absence is 
associated with a much greater likelihood of failure, regardless of incoming achieve- 
ment. The authors also examine how school practices affect students’ grades, failure 
rates and attendance. Students' grades and attendance are particularly better than ex- 
pected in schools characterized by two features — supportive relationships between 
teachers and students and a perception among students that the work they are doing in 
high school is preparing them for the future. [PDF/nc/t/c/ec/| 

Video presentation also available at the link provided below. Author Elaine Allensworth 
outlined the key findings during a joint Congressional briefing on “Turning Around Low- 
Performing High Schools: Lessons for Federal Policy from Research and Practice” aired 
on C-Span August 16, 2007. 

Link: http://ccsr.uchicaqo.edu/content/publications.php7pub id=1 1 6 


Balfanz, R., Herzog, L., & Maciver, D. J. (2007). Preventing student disengagement 
and keeping students on the graduation path in urban middie-grades schoois: 
Eariy identification and effective interventions. Educationai Psychoiogist, 
42{4), 1-13. 

Abstract: This article considers the practical, conceptual, and empirical foundations of 
an early identification and intervention system for middle-grades schools to combat stu- 
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dent disengagement and increase graduation rates in our nation's cities. Many students 
in urban schools become disengaged at the start of the middle grades, which greatly 
reduces the odds that they will eventually graduate. We use longitudinal analyses — 
following almost 13,000 students from 1996 until 2004 — to demonstrate how four pre- 
dictive indicators reflecting poor attendance, misbehavior, and course failures in sixth 
grade can be used to identify 60% of the students who will not graduate from high 
school. Fortunately, by combining effective whole-school reforms with attendance, be- 
havioral, and extra-help interventions, graduation rates can be substantially increased. 
[PDF included\ 


Heppen, J. B., & Therriault, S. B. (2008 July). Developing early warning systems to 
identify potential high school dropouts. Washington, DC: National High School 
Center. 

Abstract: This guide discusses the factors that help predict the probability that individu- 
al students will eventually drop out of high school prior to graduating and includes step- 
by-step instructions for building an early warning system. [PDF included] 

Link: http://www.betterhiqhschools.com/pubs/ews guide. asp 


Heppen, J.B., O’Cummings, M., & Bowels Therriault, S. (2008 July). Early warning 
system tool. Washington, DC: National High School Center. 

Abstract: This tool allows educators to input student-level data and automatically calcu- 
late whether individual students are on track to graduate or are at risk of dropping out. 
(Please maximize the document’s screen in Excel in order to access all the spreadsheet 
tabs in the tool.) [Excel file included] 

Link: http://www.betterhiqhschools.com/pubs/EWStool.xls 


Jerald, C. D. (2006 June). Identifying potential dropouts: Key lessons for building 
an early warning data system. Washington, DC: Achieve, Inc. 

Abstract: A white paper prepared for Staying the Course: High Standards and Im- 
proved Graduation Rates, a joint project of Achieve and Jobs for the Future, funded by 
Carnegie Corp. of New York. [PDF/nc/t/c/ec/| 

Link: http://www.achieve.orq/files/ldentifvinqPotentialDropouts.pdf and 
http://www.achieve.orq/files/FINAL-dropouts O.pdf 
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Kennelly, L., & Monrad, M. (2007 October). Approaches to dropout prevention: 
Heeding eariy warning signs with appropriate interventions. Washington, DC: 
National High School Center at the American Institutes for Research. 

Abstract: This report outlines steps that schools can take to identify at-risk students 
and provide the necessary support systems and relevant interventions to assist stu- 
dents in obtaining a high school diploma. Further, the report discusses the use of early 
warning data systems to target interventions for groups and individual students, offers a 
variety of best practice approaches undertaken by higher-performing high schools, and 
presents effective programs that are currently being implemented to stem the dropout 
problem. [PDF included\ 

Link: 

http://www.betterhiqhschools.orq/docs/NHSC ApproachestoDropoutPrevention.pdf 


Mann, D. (1989). Effective schools as a dropout prevention strategy. NASSP Bui- 
ietin, 73(518), 77-83. 

Abstract: Effective schools research and techniques are useful in developing a strategy 
for dropout prevention. Early intervention can avoid the necessity of remediation in later 
years. [PDF/nc/t/c/ec/| 


Neild, R. C., Balfanz, R., & Herzog, L. (2007). An early warning system. 

Educationai Leadership, 65(2), 28-33. 

Abstract: Part of a special issue on early intervention at every age. Schools can redi- 
rect potential dropouts toward graduation by promptly reacting to student distress sig- 
nals. Research shows that a high percentage of dropouts send distress signals in the 
middle grades and that a sixth grader with a final grade of “F” in mathematics, a final 
grade of “F” in English, attendance below 80 percent for the year, or a final unsatisfacto- 
ry behavior mark in at least one class has at least a three in four chance of dropping out 
of high school. Meanwhile, ninth graders who earned fewer than two credits or attended 
school less than 70 percent of the time had at least a 75 percent chance of dropping out 
of school. Strategies schools can use to help keep students at school are outlined. [PDF 
included] 


Pinkus, L. (2008 August). Using early-warning data to improve graduation rates: 
Closing cracks in the education system. Aiiiance for Exceiient Education Poii- 
cy Brief, 1-14. 


REL 

SOUTHEAST 


Early Warning Indicators 


6 


Abstract: This brief explores the power of early-warning data in predicting whether a 
student will drop out, offers examples of current efforts to use such data to guide sec- 
ondary school interventions across the country, and discusses the policies that can 
support these efforts. [PDF/nc/uc/ec/| 

Link: https://www.all4ed.orq/files/EWI.pdf 


Reeves, D. (2007).Teachers step up. Educational Leadership, 55(1), 87-88. 

Abstract: Jenks Public Schools in Oklahoma, winner of the 2005 Baldrige Award for 
school quality, has made remarkable progress with some of the most challenging stu- 
dents in the system by finding the most accurate early warning indicators of student 
course failures. The school does not wait for a failing grade to institute intervention 
strategies, intervention is delivered by outstanding faculty members, and intervention 
strategies include twice the classroom contact hours that had been provided in the past. 
[PDF included\ 


Roderick, M., & Camburn, E. (1999). Risk and recovery from course faiiure in the 
eariy years of high schooi. American Educational Research Journal, 55(2), 
303-343. 


Abstract: For many urban adolescents, the transition to and early years of high school 
are a time of academic difficulty and increasing school disengagement. In Chicago, over 
40% of ninth graders fail one or more major subjects in the first semester. This article 
examines patterns in the relative risk of course failure and recovery from failure over the 
first four semesters of high school in one urban school system. It examines how failure 
rates vary as a function of race, ethnicity, gender, age, and prior achievement and ex- 
amines between-school variation in student performance. Males and Hispanic students 
are particularly at risk. Few students recover from grade failure, and early failure often 
translates into poorer later performance. Schools vary widely in rates of failure and re- 
covery — variation that remains after adjusting for differences in schools' student body 
composition. Implications for further research and policy are discussed. [PDF/nc/t/c/ec/| 
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REGIONAL EDUCATIONAL LAB- SOUTHEAST March 2009 




We provide research based information on 
educationai initiatives happening nationaiiy and 
regionaiiy. The EBE Request Desk is currentiy taking 
requests for: 

- Research on a particuiar topic 

- Information on the evidence base for curricuium 
interventions or 

professionai deveiopment programs 

- Information on iarge, sponsored research projects 

- Information on southeastern state poiicies and 
programs 

For more information or to make a request, contact: 
Karia Lewis 
1.800.755.3277 
kiewis@serve.org 


The Regional Educational Laboratory (REL) - Southeast's Evidence Based Education (EBE) Request Desk is a service provided by a 
collaborative of the REL program, funded by the U.S. Department of Education's Institute of Education Sciences (lES). This response 
was prepared under a contract with lES, Contract ED-06-CO-0028, by REL-Southeast administered by the SERVE Center at the 
University of North Carolina at Greensboro. The content of the response does not necessarily reflect the views or policies of lES or the 
U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the 
U.S. Government. 
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